Prediction of anti-cancer drug synergy based on cross-matching network and cancer molecular subtypes.

At present, anti-cancer drug synergy therapy is one of the most important methods to overcome drug resistance and reduce drug toxicity in cancer treatment. High-throughput screening through deep learning can effectively improve the efficiency of discovering synergistic drugs. Nowadays, most of the existing deep learning algorithms for anti-cancer drug synergy prediction use deep neural networks and can only implicitly perform feature interaction. This study proposes a deep learning algorithm, named MolCross, which combines implicit feature interaction with explicit features to improve the accuracy of prediction of the anti-cancer drug synergy score. MolCross uses a deep autoencoder to extract features from high-dimensional input, uses the drug-specific subnetworks and cross-network to perform implicit feature interaction and explicit feature interaction respectively, and finally uses a synergy prediction network to combine the two feature interaction methods to obtain the final prediction results. We adopted a five-fold cross validation and compared MolCross with other four anti-cancer drug synergy prediction models. The results show that MolCross has better prediction performance than other models. MolCross also has good performance in terms of cross-cell line and cross-tissue type. Existing studies have demonstrated that cancer molecular subtypes have different sensitivities to targeted therapy. In this study, the features of cancer molecular subtype were introduced in the model using an embedding layer in MolCross to explore the effect of cancer molecular subtype on anti-cancer drug synergy. We also found that the cancer molecular subtype is one of the main factors affecting the synergy between drugs.

Image Intensity Variation Information for Interest Point Detection.

Interest point detection methods are gaining more attention and are widely applied in computer vision tasks such as image retrieval and 3D reconstruction. However, there still exist two main problems to be solved: (1) from the perspective of mathematical representations, the differences among edges, corners, and blobs have not been convincingly explained and the relationships among the amplitude response, scale factor, and filtering orientation for interest points have not been thoroughly explained; (2) the existing design mechanism for interest point detection does not show how to accurately obtain intensity variation information on corners and blobs. In this paper, the first- and second-order Gaussian directional derivative representations of a step edge, four common genres of corners, an anisotropic-type blob, and an isotropic-type blob are analyzed and derived. Multiple interest point characteristics are discovered. The characteristics for interest points that we obtained help us describe the differences among edges, corners, and blobs, explain why the existing interest point detection methods with multiple scales cannot properly obtain interest points from images, and present novel corner and blob detection methods. Extensive experiments demonstrate the superiority of our proposed methods in terms of detection performance, robustness to affine transformations, noise, image matching, and 3D reconstruction.

Prediction of drug-induced hepatotoxicity based on histopathological whole slide images.

Liver is an important metabolic organ in human body and is sensitive to toxic chemicals or drugs. Adverse reactions caused by drug hepatotoxicity will damage the liver and hepatotoxicity is the leading cause of removal of approved drugs from the market. Therefore, it is of great significance to identify liver toxicity as early as possible in the drug development process. In this study, we developed a predictive model for drug hepatotoxicity based on histopathological whole slide images (WSI) which are the by-product of drug experiments and have received little attention. To better represent the WSIs, we constructed a graph representation for each WSI by dividing it into small patches, taking sampled patches as nodes and calculating the correlation coefficients between node features as the edges of the graph structure. Then a WSI-level graph convolutional network (GCN) was built to effectively extract the node information of the graph and predict the toxicity. In addition, we introduced a gated attention global context vector (gaGCV) to combine the global context to make node features to contain more comprehensive information. The results validated on rat liver in vivo data from the Open TG-GATES show that the use of WSI for the prediction of toxicity is feasible and effective.

Image Feature Information Extraction for Interest Point Detection: A Comprehensive Review.

Interest point detection is one of the most fundamental and critical problems in computer vision and image processing. In this paper, we carry out a comprehensive review on image feature information (IFI) extraction techniques for interest point detection. To systematically introduce how the existing interest point detection methods extract IFI from an input image, we propose a taxonomy of the IFI extraction techniques for interest point detection. According to this taxonomy, we discuss different types of IFI extraction techniques for interest point detection. Furthermore, we identify the main unresolved issues related to the existing IFI extraction techniques for interest point detection and any interest point detection methods that have not been discussed before. The existing popular datasets and evaluation standards are provided and the performances for fifteen state-of-the-art approaches are evaluated and discussed. Moreover, future research directions on IFI extraction techniques for interest point detection are elaborated.

Image local structure information learning for fine-grained visual classification.

Learning discriminative visual patterns from image local salient regions is widely used for fine-grained visual classification (FGVC) tasks such as plant or animal species classification. A large number of complex networks have been designed for learning discriminative feature representations. In this paper, we propose a novel local structure information (LSI) learning method for FGVC. Firstly, we indicate that the existing FGVC methods have not properly considered how to extract LSI from an input image for FGVC. Then an LSI extraction technique is introduced which has the ability to properly depict the properties of different local structure features in images. Secondly, a novel LSI learning module is proposed to be added into a given backbone network for enhancing the ability of the network to find salient regions. Thirdly, extensive experiments show that our proposed method achieves better performance on six image datasets. Particularly, the proposed method performs far better on datasets with a limited number of images.

Super-resolution phase retrieval network for single-pattern structured light 3D imaging.

Structured light 3D imaging is often used for obtaining accurate 3D information via phase retrieval. Single-pattern structured light 3D imaging is much faster than multi-pattern versions. Current phase retrieval methods for single-pattern structured light 3D imaging are however not accurate enough. Besides, the projector resolution in a structured light 3D imaging system is expensive to improve due to hardware costs. To address the issues of low accuracy and low resolution of single-pattern structured light 3D imaging, this work proposes a super-resolution phase retrieval network (SRPRNet). Specifically, a phase-shifting module is proposed to extract multi-scale features with different phase shifts, and a refinement and super-resolution module is proposed to obtain refined and super-resolution phase components. After phase demodulation and unwrapping, high-resolution absolute phase is obtained. A sine shifting loss and a cosine shifting loss are also introduced to form the regularization term of the loss function. As far as can be ascertained, the proposed SRPRNet is the first network for super-resolution phase retrieval by using a single pattern, and it can also be used for standard-resolution phase retrieval. Experimental results on three datasets show that SRPRNet achieves state-of-the-art performance on 1×, 2×, and 4× super-resolution phase retrieval tasks.

Robust Blood Cell Image Segmentation Method Based on Neural Ordinary Differential Equations.

For the analysis of medical images, one of the most basic methods is to diagnose diseases by examining blood smears through a microscope to check the morphology, number, and ratio of red blood cells and white blood cells. Therefore, accurate segmentation of blood cell images is essential for cell counting and identification. The aim of this paper is to perform blood smear image segmentation by combining neural ordinary differential equations (NODEs) with U-Net networks to improve the accuracy of image segmentation. In order to study the effect of ODE-solve on the speed and accuracy of the network, the ODE-block module was added to the nine convolutional layers in the U-Net network. Firstly, blood cell images are preprocessed to enhance the contrast between the regions to be segmented; secondly, the same dataset was used for the training set and testing set to test segmentation results. According to the experimental results, we select the location where the ordinary differential equation block (ODE-block) module is added, select the appropriate error tolerance, and balance the calculation time and the segmentation accuracy, in order to exert the best performance; finally, the error tolerance of the ODE-block is adjusted to increase the network depth, and the training NODEs-UNet network model is used for cell image segmentation. Using our proposed network model to segment blood cell images in the testing set, it can achieve 95.3% pixel accuracy and 90.61% mean intersection over union. By comparing the U-Net and ResNet networks, the pixel accuracy of our network model is increased by 0.88% and 0.46%, respectively, and the mean intersection over union is increased by 2.18% and 1.13%, respectively. Our proposed network model improves the accuracy of blood cell image segmentation and reduces the computational cost of the network.

Attentive Feature Refinement Network for Single Rainy Image Restoration.

Despite the fact that great progress has been made on single image deraining tasks, it is still challenging for existing models to produce satisfactory results directly, and it often requires a single or multiple refinement stages to gradually improve the quality. However, in this paper, we demonstrate that existing image-level refinement with a stage-independent learning design is problematic with the side effect of over/under-deraining. To resolve this issue, we for the first time propose the mechanism of learning to carry out refinement on the unsatisfactory features, and propose a novel attentive feature refinement (AFR) module. Specifically, AFR is designed as a two-branched network for simultaneous rain-distribution-aware attention map learning and attention guided hierarchy-preserving feature refinement. Guided by task-specific attention, coarse features are progressively refined to better model the diversified rainy effects. By using a separable convolution as the basic component, our AFR module introduces little computation overhead and can be readily integrated into most rainy-to-clean image translation networks for achieving better deraining results. By incorporating a series of AFR modules into a general encoder-decoder network, AFR-Net is constructed for deraining and it achieves new state-of-the-art results on both synthetic and real images. Furthermore, by using AFR-Net as a teacher model, we explore the use of knowledge distillation to successfully learn a student model that is also able to achieve state-of-the-art results but with a much faster inference speed (i.e., it only takes 0.08 second to process a 512×512 rainy image). Code and pre-trained models are available at 〈 https://github.com/RobinCSIRO/AFR-Net 〉 .

Domain adaptation based self-correction model for COVID-19 infection segmentation in CT images.

The capability of generalization to unseen domains is crucial for deep learning models when considering real-world scenarios. However, current available medical image datasets, such as those for COVID-19 CT images, have large variations of infections and domain shift problems. To address this issue, we propose a prior knowledge driven domain adaptation and a dual-domain enhanced self-correction learning scheme. Based on the novel learning scheme, a domain adaptation based self-correction model (DASC-Net) is proposed for COVID-19 infection segmentation on CT images. DASC-Net consists of a novel attention and feature domain enhanced domain adaptation model (AFD-DA) to solve the domain shifts and a self-correction learning process to refine segmentation results. The innovations in AFD-DA include an image-level activation feature extractor with attention to lung abnormalities and a multi-level discrimination module for hierarchical feature domain alignment. The proposed self-correction learning process adaptively aggregates the learned model and corresponding pseudo labels for the propagation of aligned source and target domain information to alleviate the overfitting to noises caused by pseudo labels. Extensive experiments over three publicly available COVID-19 CT datasets demonstrate that DASC-Net consistently outperforms state-of-the-art segmentation, domain shift, and coronavirus infection segmentation methods. Ablation analysis further shows the effectiveness of the major components in our model. The DASC-Net enriches the theory of domain adaptation and self-correction learning in medical imaging and can be generalized to multi-site COVID-19 infection segmentation on CT images for clinical deployment.

Corner Detection Using Second-Order Generalized Gaussian Directional Derivative Representations.

Corner detection is a critical component of many image analysis and image understanding tasks, such as object recognition and image matching. Our research indicates that existing corner detection algorithms cannot properly depict the difference between edges and corners and this results in wrong corner detections. In this paper, the capability of second-order generalized (isotropic and anisotropic) Gaussian directional derivative filters to suppress Gaussian noise is evaluated. The second-order generalized Gaussian directional derivative representations of step edge, L-type corner, Y- or T-type corner, X-type corner, and star-type corner are investigated and obtained. A number of properties for edges and corners are discovered which enable us to propose a new image corner detection method. Finally, the criteria on detection accuracy and average repeatability under affine image transformation, JPEG compression, and noise degradation, and the criteria on region repeatability are used to evaluate the proposed detector against nine state-of-the-art methods. The experimental results show that our proposed detector outperforms all the other tested detectors.

Robust Tensor Decomposition for Image Representation Based on Generalized Correntropy.

Traditional tensor decomposition methods, e.g., two dimensional principal component analysis and two dimensional singular value decomposition, that minimize mean square errors, are sensitive to outliers. To overcome this problem, in this paper we propose a new robust tensor decomposition method using generalized correntropy criterion (Corr-Tensor). A Lagrange multiplier method is used to effectively optimize the generalized correntropy objective function in an iterative manner. The Corr-Tensor can effectively improve the robustness of tensor decomposition with the existence of outliers without introducing any extra computational cost. Experimental results demonstrated that the proposed method significantly reduces the reconstruction error on face reconstruction and improves the accuracies on handwritten digit recognition and facial image clustering.

Cascaded Attention Guidance Network for Single Rainy Image Restoration.

Restoring a rainy image with raindrops or rainstreaks of varying scales, directions, and densities is an extremely challenging task. Recent approaches attempt to leverage the rain distribution (e.g., location) as prior to generate satisfactory results. However, concatenation of a single distribution map with the rainy image or with intermediate feature maps is too simplistic to fully exploit the advantages of such priors. To further explore this valuable information, an advanced cascaded attention guidance network, dubbed as CAG-Net, is formulated and designed as a three-stage model. In the first stage, a multitask learning network is constructed for producing the attention map and coarse de-raining results simultaneously. Subsequently, the coarse results and the rain distribution map are concatenated and fed to the second stage for results refinement. In this stage, the attention map generation network from the first stage is used to formulate a novel semantic consistency loss for better detail recovery. In the third stage, a novel pyramidal "whereand- how" learning mechanism is formulated. At each pyramid level, a two-branch network is designed to take the features from previous stages as inputs to generate better attention-guidance features and de-raining features, which are then combined via a gating scheme to produce the final de-raining results. Moreover, the uncertainty maps are also generated in this stage for more accurate pixel-wise loss calculation. Extensive experiments are carried out for removing raindrops or rainstreaks from both synthetic and real rainy images, and CAG-Net is demonstrated to produce significantly better results than state-of-the-art models. Code will be publicly available after paper acceptance.

Deep learning based HEp-2 image classification: A comprehensive review.

Classification of HEp-2 cell patterns plays a significant role in the indirect immunofluorescence test for identifying autoimmune diseases in the human body. Many automatic HEp-2 cell classification methods have been proposed in recent years, amongst which deep learning based methods have shown impressive performance. This paper provides a comprehensive review of the existing deep learning based HEp-2 cell image classification methods. These methods perform HEp-2 image classification at two levels, namely, cell-level and specimen-level. Both levels are covered in this review. At each level, the methods are organized with a deep network usage based taxonomy. The core idea, notable achievements, and key strengths and weaknesses of each method are critically analyzed. Furthermore, a concise review of the existing HEp-2 datasets that are commonly used in the literature is given. The paper ends with a discussion on novel opportunities and future research directions in this field. It is hoped that this paper would provide readers with a thorough reference of this novel, challenging, and thriving field.

RA-UNet: A Hybrid Deep Attention-Aware Network to Extract Liver and Tumor in CT Scans.

Automatic extraction of liver and tumor from CT volumes is a challenging task due to their heterogeneous and diffusive shapes. Recently, 2D deep convolutional neural networks have become popular in medical image segmentation tasks because of the utilization of large labeled datasets to learn hierarchical features. However, few studies investigate 3D networks for liver tumor segmentation. In this paper, we propose a 3D hybrid residual attention-aware segmentation method, i.e., RA-UNet, to precisely extract the liver region and segment tumors from the liver. The proposed network has a basic architecture as U-Net which extracts contextual information combining low-level feature maps with high-level ones. Attention residual modules are integrated so that the attention-aware features change adaptively. This is the first work that an attention residual mechanism is used to segment tumors from 3D medical volumetric images. We evaluated our framework on the public MICCAI 2017 Liver Tumor Segmentation dataset and tested the generalization on the 3DIRCADb dataset. The experiments show that our architecture obtains competitive results.

[Effect of Down-Regulating the CD59 by RNAi Lentivirus on the Expression of Acute T-lineage Leukemia Jurkat Cell Line].

OBJECTIVE: To analyze the effect of down-regulating the CD59 gene expression by RNAi lentivirus as vector on Jurkat cell line of acute T-lineage leukemia. METHODS: The expression of CD59 in Jurkat cell line of acute T-line leukemia was induced to decrease by RNAi lentivirus as vector. The transfection of RNA lentivirus and the localization of CD59 molecule were analyzed by laser confocal technique. The relative expression of CD59 gene in blank control, negative control and RNAi lentivirus transfected group was detected by real-time fluorescence quantitative PCR, and the enzyme-linked immunosorbent assay was used to detect the expression of TNF-ß and IL-3 in supernatants of cultured cells in 3 groups. The expression levels of apoptosis-related molecules including Caspase-3, Survivin, BCL-2 and BCL-2-associated X protein (BAX) were measured by Western blot. RESULTS: The transfection efficiency for Jurkat cells was higher than 90%. CD59 was mainly located on the cell membrane. Compared with the blank control group and the negative control group, the expression level of CD59 mRNA and protein in the RNAi lentivirus transfected group significantly decreased (P<0.05). Compared with the blank control group and the negative control group, the expression of TNF-ß and IL-3 in the RNAi lentivirus transfected group were significantly higher and lower (P<0.05) respectively. The expression levels of Survivin and BCL-2 in the RNAi lentivirus transfected group were significantly lower than those in the blank control group and the negative control group, while the expression levels of Caspase-3 and BAX in the RNAi lentivirus transfected group were significantly higher than those in the blank control group and the negative control group (P< 0.05). CONCLUSION: The down-regulation of CD59 gene expression induced by RNAi lenti-virus can decrease the expression of proliferation and differentiation-promoting molecule such as IL-3 and increase the expression of TNF-related factor in Jurkat cell line of acute T-lineage leukemia, which also can increase the expression of apoptosis-related proteins such as Caspase-3 and BAX, and decrease the expression of anti-apoptosis-related proteins such as Survivin and BCL-2.

Chord Bunch Walks for Recognizing Naturally Self-Overlapped and Compound Leaves.

Effectively describing and recognizing leaf shapes under arbitrary variations, particularly from a large database, remains an unsolved problem. In this research, we attempted a new strategy of describing leaf shapes by walking and measuring along a bunch of chords that pass through the shape. A novel chord bunch walks (CBW) descriptor is developed through the chord walking behavior that effectively integrates the shape image function over the walked chord to reflect both the contour features and the inner properties of the shape. For each contour point, the chord bunch groups multiple pairs of chords to build a hierarchical framework for a coarse-to-fine description that can effectively characterize not only the subtle differences among leaf margin patterns but also the interior part of the shape contour formed inside a self-overlapped or compound leaf. Instead of using optimal correspondence based matching, a Log-Min distance that encourages one-to-one correspondences is proposed for efficient and effective CBW matching. The proposed CBW shape analysis method is invariant to rotation, scaling, translation, and mirror transforms. Five experiments, including image retrieval of compound leaves, image retrieval of naturally self-overlapped leaves, and retrieval of mixed leaves on three large scale datasets, are conducted. The proposed method achieved large accuracy increases with low computational costs over the state-of-the-art benchmarks, which indicates the research potential along this direction.

Discrete Curvature Representations for Noise Robust Image Corner Detection.

Image corner detection is very important in the fields of image analysis and computer vision. Curvature calculation techniques are used in many contour-based corner detectors. We identify that existing calculation of curvature is sensitive to local variation and noise in the discrete domain and does not perform well when corners are closely located. In this paper, discrete curvature representations of single and double corner models are investigated and obtained. A number of model properties have been discovered, which help us detect corners on contours. It is shown that the proposed method has a high corner resolution (the ability to accurately detect neighboring corners), and a corresponding corner resolution constant is also derived. Meanwhile, this method is less sensitive to any local variations and noise on the contour; and false corner detection is less likely to occur. The proposed detector is compared with seven state-of-the-art detectors. Three test images with ground truths are used to assess the detection capability and localization accuracy of these methods in cases with noise-free and different noise levels; 24 images with various scenes without ground truths are used to evaluate their repeatability under affine transformation, JPEG compression, and noise degradations. The experimental results show that our proposed detector attains a better overall performance.

Supervised Brain Tumor Segmentation Based on Gradient and Context-Sensitive Features.

Gliomas have the highest mortality rate and prevalence among the primary brain tumors. In this study, we proposed a supervised brain tumor segmentation method which detects diverse tumoral structures of both high grade gliomas and low grade gliomas in magnetic resonance imaging (MRI) images based on two types of features, the gradient features and the context-sensitive features. Two-dimensional gradient and three-dimensional gradient information was fully utilized to capture the gradient change. Furthermore, we proposed a circular context-sensitive feature which captures context information effectively. These features, totally 62, were compressed and optimized based on an mRMR algorithm, and random forest was used to classify voxels based on the compact feature set. To overcome the class-imbalanced problem of MRI data, our model was trained on a class-balanced region of interest dataset. We evaluated the proposed method based on the 2015 Brain Tumor Segmentation Challenge database, and the experimental results show a competitive performance.

Cell Segmentation Based on FOPSO Combined With Shape Information Improved Intuitionistic FCM.

Fuzzy c-means (FCM) clustering algorithms have been proved to be effective image segmentation techniques. However, FCM clustering algorithms are sensitive to noises and initialization. They cannot effectively segment cell images with inhomogeneous gray value distributions and complex touching cells. Aiming to overcome these disadvantages, this paper proposes a cell image segmentation algorithm using fractional-order velocity based particle swarm optimization (FOPSO) combined with shape information improved intuitionistic FCM (SI-IFCM) clustering. Iterations are carried out between FOPSO and SI-IFCM to achieve final cell segmentation. Experimental results demonstrate that the proposed algorithm has advantages on cell image segmentation, with the highest recall (90.25%) and lowest false discovery rate (0.28%) compared with the state-of-the-art algorithms.

Design of a Clinical Decision Support System for Predicting Erectile Dysfunction in Men Using NHIRD Dataset.

Erectile dysfunction (ED) affects millions of men worldwide. Men with ED generally complain failure to attain or maintain an adequate erection during sexual activity. The prevalence of ED is strongly correlated with age, affecting about 40% of men at age 40 and nearly 70% at age 70. A variety of chronic diseases, including diabetes, ischemic heart disease, congestive heart failure, hypertension, depression, chronic renal failure, obstructive sleep apnea, prostate disease, gout, and sleep disorder, were reported to be associated with ED. In this study, data retrieved from a subset of the National Health Insurance Research Database of Taiwan were used for designing the clinical decision support system (CDSS) for predicting ED incidences in men. The positive cases were male patients aged 20-65 who were diagnosed with ED between January 2000 and December 2010 confirmed by at least three outpatient visits or at least one inpatient visit, while the negative cases were randomly selected from the database without a history of ED and were frequency (1:1), age, and index year matched with the ED patients. Data of a total of 2832 ED patients and 2832 non-ED patients, each consisting of 41 features including index age, 10 comorbidities, and 30 other comorbidity-related variables, were retrieved for designing the predictive models. Integrated genetic algorithm and support vector machine was adopted to design the CDSSs with two experiments of independent training and testing (ITT) conducted to verify their effectiveness. In the 1st ITT experiment, data extracted from January 2000 till December 2005 (61.51%, 1742 positive cases and 1742 negative cases) were used for training and validating and the data retrieved from January 2006 till December 2010 were used for testing (38.49%), whereas in the 2nd ITT experiment, data in the training set (77.78%) were extracted from January 2000 till Deceber 2007 and those in the testing set (22.22%) were retrieved afterward. Tenfold cross validation and three different objective functions were adopted for obtaining the optimal models with best predictive performance in the training phase. The testing results show that the CDSSs achieved a predictive performance with accuracy, sensitivity, specificity, g-mean, and area under ROC curve of 74.72%-76.65%, 72.33%-83.76%, 69.54%-77.10%, 0.7468-0.7632, and 0.766-0.817, respectively. In conclusion, the CDSSs designed based on cost-sensitive objective functions as well as salient comorbidity-related features achieve satisfactory predictive performance for predicting ED incidences.